We study the horizontal distribution of zeros of $\zeta'(s)$ which are denoted as $\rho'=\beta'+i\gamma'$. We assume the Riemann hypothesis which implies $\beta'\geqslant1/2$ for any non-real zero $\rho'$, equality being possible only at a multiple zero of $\zeta(s)$. In this paper we prove that $\liminf(\beta'-1/2)\log\gamma'\not=0$ if and only if for any $c>0$ and $s=\sigma+it$ with $|\sigma-1/2|0$ and $s=\sigma+it$ ($t\geqslant10$), we have $$ \log\zeta(s)=O(\frac{(\log t)^{2-2\sigma}}{\log\log t}) $$ uniformly for $1/2+c/\log t\leqslant\sigma\leqslant\sigma_1<1$.